The present invention relates to a image-information storage regenerating device, and more particularly to a video communication system which can collect and store multiplexed data of audio/video and other information from video data communication terminals (e.g., visual telephone sets) according to the recommendation by ITU (International Telecommunication Union) and which allows users to regenerate the stored data through their terminal sets. More particularly, the present invention is related to a video data storage control device for regenerating a video data file stored in a storing/regenerating center (storage device) for video data, and more particularly to a video data regeneration control device which is capable of transmitting stored coded video data from a video data storage and regeneration center (storage device) to an audio-visual (AV) terminal and enabling the terminal to decode the received coded video data and reproduce images thereat, and which is also capable of transmitting stored therein coded video data to a receiving terminal and enabling the terminal to decode the received coded video data and reproduce therefrom the video data.
Recently, besides conventional interactive communication services using telephones and facsimiles, such a communication service has been provided that audio data stored at a center of communication network can be always regenerated through users' terminal sets. On the other hand, with improvement and development of digital communications networks, e.g., ISDN (Integrated Services Digital Network), it has been possible to use video communication services, e.g., visual telephone service. On such background, there is an increasing demand for video delivery services using a database of video information. Such communication services have been locally realized for practical use.
Recently, with improvement and development of digital transmission channels and advancement of image processing technique, it has been expected to develop a method allowing one to effectively use image-information services over high-speed digital transmission networks, a representative of which is ISDN (Integrated Services Digital Network), by applying advanced high-speed digital-signal processing and related LSI technology. Representative image communication services such as visual telephone services and video conference services have been already realized and practically utilized. Lately, such an information delivery service for supplying information to many unspecified users at unspecified times has been expected to appear in practice as a new business.
To supply information to many unspecified users, it is necessary to store information in a storage (recording device), e.g., a host center, and to provide a control system for regenerating information at the many unspecified audio-visual terminals and storing information from the audio-visual terminals into the storage. This system is represented, for example, by an image-information regeneration control device that controls regeneration of image-information from the host center to the audio-visual terminals.
In a conventional image-information regeneration control device, coded image-information is stored with screen (frame) numbers and is transmitted with the same screen (frame) numbers to an audio-visual terminal at which the coded image-information is then decoded and regenerated according to the screen numbers.
The image-information communication service may often transmit a interactive-mode screen (e.g., a service menu or guidance for user) and coded data of different object images in succession to a user terminal and continuously regenerate them at the terminal.
In a conventional image-information transfer system, a center control unit negotiates with a terminal as to decoding capability of the terminal before transmitting coded image-information thereto and, only in case of the terminal having a sufficient decoding capability, it reads and transmits the coded image-information stored together with their screen (frame) numbers to the terminal.
Japanese laid-open patent publication No. 62-188579 describes a stored video data regenerating device for use in a packet communication network for transmitting moving picture information from a moving picture database, which converts video data into video signals adapted to be easily received by user's terminal sets by absorbing variations of delay in transmission of packets over the packet communications network and allows user to get desired video information from the data base at any desired time.
Japanese laid-open patent publication No. 3-132128 describes a stored video data regenerating device that allows for storing messages of video and audio data composed of still-pictures and recorded pictures inputted from users through their terminal devices and allows each user to reproduce a message corresponding to his request on a display screen of his terminal.
Japanese laid-open patent publication No. 4-79655 describes a stored video data regenerating device that has means to register conditions of collecting information from a user's terminal having functions for inputting and outputting video/audio and characters, and that calls the terminal according to the registered information and automatically receives information and stores the collected information as requested by a client.
As a conventional stored image regenerating device there may be designated a device that is described in a reference paper titled "Prototyping of PC-Videophone with Built-in Codec based on Open Architecture" (by Ichizo KIMURA and 7 et al.), the journal of the Institute of Image Electronics Engineers of Japan, 1993, Vol. 22, No. 4, pp. 355-367.
The above-mentioned reference describes how to store and regenerate audio/video data, indicating two possible methods: the first method is directed to store the multiplexed audio/video data as unchanged and the second method is directed to disassemble multiplexed data and store audio/video data as separated from each other. However, the conventional systems prefer the first method from the view point of easier processing of multiplexed data at a center device and describes necessary functions of this method and how to realize said functions.
The present invention pointed out problems of the conventional method of storing audio/video as multiplexed data and provides a novel method that separates media, particularly video (image-information), and stores them as separated from each other in order to solve the problems involved in the conventional systems. Furthermore, the present invention discloses that an excellent video communication system can be constructed by applying therein several new techniques. Practically, the data storage and regenerating device according to the present invention is featured by new functions such as data management by frames, synchronizing by time stamps and adjustment of information amount by inserting fill bits, and which realizes, thereby, so flexible and so high leveled functions that can not be attained by the conventional multiplexed data storing system.
Accordingly, it may be clearly understood that the present invention in its basic conception quite differs from the conventional systems, and it also discloses novel and useful art to solve the problems of the conventional systems.
As described above, the conventional system stores data as multiplexed by the method adopted as an international standard. The data display at the terminal device is, therefore, conducted usually after synchronization of a FEC-frame (forward error correction frame) is established. Namely, an image can not immediately be displayed. The initial. Image is not always encoded by intra-frame mode and may not be self-contained for decoding. Furthermore, the terminal may not well reproduce images because of occurrence of underflow or overflow of its buffer, which may result from that the Storing buffer condition differs from the regenerating buffer condition.
The video data are stored under the control of encoded information amount according to buffer condition of a terminal. The conventional method can not assure the same condition of buffer of a terminal device when storing data and when regenerating data. Any terminal may encounter overflow or underflow of its buffer when regenerating stored video data. Furthermore, in the conventional video regenerating device, such a problem may arise that stored video data can not be regenerated at a terminal if the channel transmission rate to which the terminal is adapted is not compatible with the channel rate at which the data were stored. For instance, said problem may occur when video data stored by a terminal adapted only to a channel rate of B using a basic interface of ISDN is regenerated at a terminal adapted only to a channel rate of 2B.
In video data storing, a transmitting AV-terminal according to the recommendation H.261 encodes successive objective frames by motion compensative inter-frame prediction and sends the encoded video data. When an video data storing center starts storing operation, coded video data by screens, for example, B.sub.o -B.sub.n are read-out and stored in a storage such as a host machine. In this case, the screen B.sub.o is encoded by prediction with reference to the frame B.sub.-1.
In image reproduction, the coded video data, e.g., frames A.sub.-3 -A.sub.1 and B.sub.o -B.sub.n stored subsequently to the screen A.sub.-1 are successively read from the video data storage center by the host machine and sent to an AV-terminal. These data maintain an integrity as successive coded video frames and they are, therefore, normally decoded at the AV-terminal.
While the screen B.sub.o is, however, predicted with reference to the screen B.sub.-1 at the time of data storing (i.e., coding), it will be decoded with reference to the screen A.sub.-1 that relates to a preceding object in the order of coded video data. Namely, the screen B.sub.o is predicted according to different screens at coding and decoding. As the result, the screen B.sub.o can not normally be reproduced and displayed at the AV-terminal. Furthermore, this may affect the subsequent screens since screens B.sub.1 -B.sub.n have been dependently coded in succession by inter-frame prediction. The desplayed screen videos may be damaged.
In the conventional image-information regeneration control device, as described above, a screen number (frame number) of video data (frame data) to be transmitted and received by an audio-visual terminal is the same as that of the corresponding video data stored in a storage. Namely, the stored video data with its unchanged screen (frame) number are transmitted to the terminal. For example, when video data are stored in the order of screen (frame) numbers "10, 12, 15, 17 . . . ", corresponding video data to be transmitted are given the same frame numbers "10, 12, 15, 17 . . . ". Accordingly, when such video data are regenerated in succession to a currently displayed image or video data of different object, a discrepancy of screen numbers may occur at a junction between two different image sequences.
For instance, in case of continuously displaying the stored video data in succession to a sequence of screen images, jointed screen (frame) numbers are described as "screen-image (1), screen-image (3), screen-image (5), video data (10), video data (12), video data(15) . . . " which is transmitted without any modification. This causes the audio-visual terminal to regenerate the received video data in the order of their screen (frame) numbers with a discrepancy of screen (frame) numbers between "screen-image (5)" and "video data (10)", i.e., a frame interval corresponding to a split of image sequences. In this case, video data behind the frame interval is regenerated with a delay time.
As described above, the conventional image-information regeneration control device has such a drawback that when a screen-image and/or different kinds of video data are successively transmitted, discontinuity of screen numbers occurs at a junction of the transmitted video data between the last screen (frame) of a preceding video data and the top screen (frame) of a subsequent video data, causing a delay time between the preceding video data and the succeeding video data when they are regenerated at the terminal.
The conventional image-information regeneration control device has to first negotiate with a receiving terminal whether the terminal has sufficient capability to decode the coded video data stored in the device or not. When the decoding capability of the terminal is insufficient, i.e., a minimal frame interval of video data to be decoded by the terminal is, for example, 2/29.97 seconds (hereinafter described as 2/30 sec.) which is more than a frame interval of the coded video data stored in a storage of the device, which stores the video data encoded at a frame interval 1/29.97 seconds (hereinafter described as 1/30 sec.) according to a coding method based upon recommendation H.261 of ITU (International Telecommunications Union), screen numbers (frame numbers) of the video data(frame data) to be received by the terminal must be "0, 2, 4, 6 . . . " which differ from screen numbers "0, 1, 2, 3 . . . " of the stored coded video data. Consequently, the center control device can not supply the terminal with the coded video data as be unchanged.
The center control device may transmit stored video data only by odd number frames (1, 3, 5 . . . ) to get frame numbers "0, 2, 4, 6 . . . ", but the terminal can not correctly regenerate images because information of each frame is coded as closely related with information of precedent and subsequent frames by inter-frame prediction coding.
As described above, the conventional video data regeneration control device involves a problem that its center control device can not communicate with a terminal not having sufficient decoding capability.
As described above, the conventional multiplexed data storing and regenerating device separates multiplexed data into audio/video and other information, processes respective data and stores or regenerates the data, resulting in arising of a differential delay between audio/video data at the time of their regeneration.